{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n",
        "\n",
        "\n",
        "\n",
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/spark_nlp_utilities/NLU_utils_for_Spark_NLP.ipynb)\n",
        "# NLU utilities for Spark NLP\n",
        "This notebook showcases various utils provided for Spark NLP by NLU"
      ],
      "metadata": {
        "id": "CUpEfC3JcETa",
        "pycharm": {
          "name": "#%% md\n"
        }
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Colab Setup"
      ],
      "metadata": {
        "id": "11sKij2tcQKo",
        "pycharm": {
          "name": "#%% md\n"
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -q johnsnowlabs"
      ],
      "metadata": {
        "id": "WN9uZX9z2KDU",
        "pycharm": {
          "name": "#%%\n"
        }
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "print('Please Upload your John Snow Labs License using the button below')\n",
        "license_keys = files.upload()"
      ],
      "metadata": {
        "id": "Ke87QahOJQpM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from johnsnowlabs import nlp\n",
        "\n",
        "# After uploading your license run this to install all licensed Python Wheels and pre-download Jars the Spark Session JVM\n",
        "nlp.install()"
      ],
      "metadata": {
        "id": "p_-OAEtMJmD9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from johnsnowlabs import nlp\n",
        "spark=nlp.start()"
      ],
      "metadata": {
        "id": "B_xUxj9QJpjX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## nlu.viz(pipe,data)\n",
        "\n",
        "Visualize input data with an already configured Spark NLP pipeline,  \n",
        "for Algorithms of type (Ner,Assertion, Relation, Resolution, Dependency)  \n",
        "using [Spark NLP Display](https://nlp.johnsnowlabs.com/docs/en/display)  \n",
        "Automatically infers applicable viz type and output columns to use for visualization.  \n",
        "\n",
        "\n",
        "\n",
        "If a pipeline has multiple models candidates that can be used for a viz,  \n",
        "the first Annotator that is vizzable will be used to create viz.  \n",
        "You can specify which type of viz to create with the viz_type parameter  \n",
        "  \n",
        "Output columns to use for the viz are automatically deducted from the pipeline, by using the  \n",
        "first annotator that provides the correct output type for a specific viz.  \n",
        "You can specify which columns to use for a viz by using the  \n",
        "corresponding ner_col, pos_col, dep_untyped_col, dep_typed_col, resolution_col, relation_col, assertion_col, parameters.\n"
      ],
      "metadata": {
        "id": "GpG5Bn3JarDD",
        "pycharm": {
          "name": "#%% md\n"
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from johnsnowlabs import nlp\n",
        "\n",
        "ade_pipeline = nlp.PretrainedPipeline('explain_clinical_doc_ade', 'en', 'clinical/models')\n",
        "\n",
        "text = \"\"\"I have an allergic reaction to vancomycin.\n",
        "    My skin has be itchy, sore throat/burning/itchy, and numbness in tongue and gums.\n",
        "    I would not recommend this dr\t- new conversion tool\"\"\"\n",
        "nlp.nlu.viz(ade_pipeline,text)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "id": "gX5c-KFAFsO1",
        "outputId": "ba32dcd3-935f-4e28-cfac-5c5068918002",
        "pycharm": {
          "name": "#%%\n"
        }
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "explain_clinical_doc_ade download started this may take some time.\n",
            "Approx size to download 462.6 MB\n",
            "[OK!]\n",
            "Warning::Spark Session already created, some configs may not take.\n",
            "Warning::Spark Session already created, some configs may not take.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n",
              "    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n",
              "    \n",
              "    .spark-nlp-display-scroll-entities {\n",
              "        border: 1px solid #E7EDF0;\n",
              "        border-radius: 3px;\n",
              "        text-align: justify;\n",
              "        \n",
              "    }\n",
              "    .spark-nlp-display-scroll-entities span {  \n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        color: #536B76;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "    }\n",
              "    \n",
              "    .spark-nlp-display-entity-wrapper{\n",
              "    \n",
              "        display: inline-grid;\n",
              "        text-align: center;\n",
              "        border-radius: 4px;\n",
              "        margin: 0 2px 5px 2px;\n",
              "        padding: 1px\n",
              "    }\n",
              "    .spark-nlp-display-entity-name{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "        \n",
              "        background: #f1f2f3;\n",
              "        border-width: medium;\n",
              "        text-align: center;\n",
              "        \n",
              "        font-weight: 400;\n",
              "        \n",
              "        border-radius: 5px;\n",
              "        padding: 2px 5px;\n",
              "        display: block;\n",
              "        margin: 3px 2px;\n",
              "    \n",
              "    }\n",
              "    .spark-nlp-display-entity-type{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        color: #ffffff;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "        \n",
              "        text-transform: uppercase;\n",
              "        \n",
              "        font-weight: 500;\n",
              "\n",
              "        display: block;\n",
              "        padding: 3px 5px;\n",
              "    }\n",
              "    \n",
              "    .spark-nlp-display-entity-resolution{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        color: #ffffff;\n",
              "        font-family: 'Vistol Regular', sans-serif !important;\n",
              "        \n",
              "        text-transform: uppercase;\n",
              "        \n",
              "        font-weight: 500;\n",
              "\n",
              "        display: block;\n",
              "        padding: 3px 5px;\n",
              "    }\n",
              "    \n",
              "    .spark-nlp-display-others{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "        \n",
              "        font-weight: 400;\n",
              "    }\n",
              "\n",
              "</style>\n",
              " <span class=\"spark-nlp-display-others\" style=\"background-color: white\">I have an </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5B7C25B3\"><span class=\"spark-nlp-display-entity-name\">allergic reaction </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #5B7C25FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> to vancomycin. <br>    My skin has be </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5B7C25B3\"><span class=\"spark-nlp-display-entity-name\">itchy </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #5B7C25FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5B7C25B3\"><span class=\"spark-nlp-display-entity-name\">sore throat/burning/itchy </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #5B7C25FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5B7C25B3\"><span class=\"spark-nlp-display-entity-name\">numbness in tongue and gums </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #5B7C25FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. <br>    I would not recommend this dr\t- new conversion tool</span></div>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## nlu.to_pretty_df(pipe,data)\n",
        "\n",
        "Annotates a Pandas Dataframe/Pandas Series/Numpy Array/Spark DataFrame/Python List strings /Python String  \n",
        "with given Spark NLP pipeline, which is assumed to be complete and runnable and returns it in a pythonic pandas dataframe format.\n",
        "\n",
        "\n",
        "Annotators are grouped internally by NLU into output levels `token`,`sentence`, `document`,`chunk` and `relation`\n",
        "Same level annotators output columns are zipped and exploded together to create  the final output df.\n",
        "Additionally, most keys from the metadata dictionary in the result annotations will be collected and expanded into their own columns in the resulting Dataframe, with special handling for Annotators that encode multiple metadata fields inside of one, seperated by strings like `|||`   or `:::`.\n",
        "Some columns are omitted from metadata to reduce total amount of output columns, these can be re-enabled by setting `metadata=True`\n",
        "\n",
        "For a given pipeline output level is automatically set to the last anntators output level by default.\n",
        "This can be changed by defining `to_preddty_df(pipe,text,output_level='my_level'` for levels `token`,`sentence`, `document`,`chunk` and `relation` ."
      ],
      "metadata": {
        "id": "jqXbQZoZa4fA",
        "pycharm": {
          "name": "#%% md\n"
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# works with Pipeline, LightPipeline, PipelineModel, List[Annotator]\n",
        "\n",
        "text = \"\"\"I have an allergic reaction to vancomycin.\n",
        "    My skin has be itchy, sore throat/burning/itchy, and numbness in tongue and gums.\n",
        "    I would not recommend this drug to anyone, especially since I have never had such an adverse reaction to any other medication.\"\"\"\n",
        "ade_assert_cols = ['assertion', 'entities_ner_chunks_ade_assertion',\t 'entities_ner_chunks_ade_assertion_class','assertion_confidence']\n",
        "df = nlp.to_pretty_df(ade_pipeline,text)\n",
        "df[ade_assert_cols]\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 241
        },
        "id": "amg15blSGANo",
        "outputId": "851139e4-3ee6-43e9-cddc-2081596ea650",
        "pycharm": {
          "name": "#%%\n"
        }
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Warning::Spark Session already created, some configs may not take.\n",
            "Warning::Spark Session already created, some configs may not take.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  assertion entities_ner_chunks_ade_assertion  \\\n",
              "0   present                 allergic reaction   \n",
              "0   present                             itchy   \n",
              "0   present         sore throat/burning/itchy   \n",
              "0   present       numbness in tongue and gums   \n",
              "0       NaN                               NaN   \n",
              "\n",
              "  entities_ner_chunks_ade_assertion_class assertion_confidence  \n",
              "0                                     ADE               0.9532  \n",
              "0                                     ADE               0.9641  \n",
              "0                                     ADE               0.9184  \n",
              "0                                     ADE               0.9998  \n",
              "0                                     NaN                  NaN  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-5197507d-eba3-4ea7-a17e-38693b4c2d5e\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>assertion</th>\n",
              "      <th>entities_ner_chunks_ade_assertion</th>\n",
              "      <th>entities_ner_chunks_ade_assertion_class</th>\n",
              "      <th>assertion_confidence</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>allergic reaction</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9532</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>itchy</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9641</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>sore throat/burning/itchy</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9184</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>numbness in tongue and gums</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9998</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5197507d-eba3-4ea7-a17e-38693b4c2d5e')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-5197507d-eba3-4ea7-a17e-38693b4c2d5e button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-5197507d-eba3-4ea7-a17e-38693b4c2d5e');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-4375cfcc-78f8-436b-a00a-01c5aa835b9d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4375cfcc-78f8-436b-a00a-01c5aa835b9d')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-4375cfcc-78f8-436b-a00a-01c5aa835b9d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## nlu.to_nlu_pipe(pipe)\n",
        "\n",
        "Convert a pipeline or list of annotators into a NLU pipeline making `.predict()` and `.viz()` avaiable for every Spark NLP pipeline.\n",
        "Assumes the pipeline is already runnable.\n"
      ],
      "metadata": {
        "id": "KI5u-6aia8bo",
        "pycharm": {
          "name": "#%% md\n"
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "text = \"\"\"I have an allergic reaction to vancomycin.\n",
        "    My skin has be itchy, sore throat/burning/itchy, and numbness in tongue and gums.\n",
        "    I would not recommend this drug to anyone, especially since I have never had such an adverse reaction to any other medication.\"\"\"\n",
        "nlu_pipe = nlp.to_nlu_pipe(ade_pipeline)\n",
        "nlu_pipe.viz(text)\n",
        "nlu_pipe.predict(text)[ade_assert_cols]\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 467
        },
        "id": "xuZL1uTmF5SJ",
        "outputId": "df5ffc84-ccde-467b-87a1-ca9214f85050",
        "pycharm": {
          "name": "#%%\n"
        }
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Warning::Spark Session already created, some configs may not take.\n",
            "Warning::Spark Session already created, some configs may not take.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n",
              "    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n",
              "    \n",
              "    .spark-nlp-display-scroll-entities {\n",
              "        border: 1px solid #E7EDF0;\n",
              "        border-radius: 3px;\n",
              "        text-align: justify;\n",
              "        \n",
              "    }\n",
              "    .spark-nlp-display-scroll-entities span {  \n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        color: #536B76;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "    }\n",
              "    \n",
              "    .spark-nlp-display-entity-wrapper{\n",
              "    \n",
              "        display: inline-grid;\n",
              "        text-align: center;\n",
              "        border-radius: 4px;\n",
              "        margin: 0 2px 5px 2px;\n",
              "        padding: 1px\n",
              "    }\n",
              "    .spark-nlp-display-entity-name{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "        \n",
              "        background: #f1f2f3;\n",
              "        border-width: medium;\n",
              "        text-align: center;\n",
              "        \n",
              "        font-weight: 400;\n",
              "        \n",
              "        border-radius: 5px;\n",
              "        padding: 2px 5px;\n",
              "        display: block;\n",
              "        margin: 3px 2px;\n",
              "    \n",
              "    }\n",
              "    .spark-nlp-display-entity-type{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        color: #ffffff;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "        \n",
              "        text-transform: uppercase;\n",
              "        \n",
              "        font-weight: 500;\n",
              "\n",
              "        display: block;\n",
              "        padding: 3px 5px;\n",
              "    }\n",
              "    \n",
              "    .spark-nlp-display-entity-resolution{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        color: #ffffff;\n",
              "        font-family: 'Vistol Regular', sans-serif !important;\n",
              "        \n",
              "        text-transform: uppercase;\n",
              "        \n",
              "        font-weight: 500;\n",
              "\n",
              "        display: block;\n",
              "        padding: 3px 5px;\n",
              "    }\n",
              "    \n",
              "    .spark-nlp-display-others{\n",
              "        font-size: 14px;\n",
              "        line-height: 24px;\n",
              "        font-family: 'Montserrat', sans-serif !important;\n",
              "        \n",
              "        font-weight: 400;\n",
              "    }\n",
              "\n",
              "</style>\n",
              " <span class=\"spark-nlp-display-others\" style=\"background-color: white\">I have an </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C7B894B3\"><span class=\"spark-nlp-display-entity-name\">allergic reaction </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #C7B894FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> to vancomycin. <br>    My skin has be </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C7B894B3\"><span class=\"spark-nlp-display-entity-name\">itchy </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #C7B894FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C7B894B3\"><span class=\"spark-nlp-display-entity-name\">sore throat/burning/itchy </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #C7B894FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C7B894B3\"><span class=\"spark-nlp-display-entity-name\">numbness in tongue and gums </span><span class=\"spark-nlp-display-entity-type\">ADE</span><span class=\"spark-nlp-display-entity-resolution\" style=\"background-color: #C7B894FF\">present </span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. <br>    I would not recommend this drug to anyone, especially since I have never had such an adverse reaction to any other medication.</span></div>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  assertion entities_ner_chunks_ade_assertion  \\\n",
              "0   present                 allergic reaction   \n",
              "0   present                             itchy   \n",
              "0   present         sore throat/burning/itchy   \n",
              "0   present       numbness in tongue and gums   \n",
              "0       NaN                               NaN   \n",
              "\n",
              "  entities_ner_chunks_ade_assertion_class assertion_confidence  \n",
              "0                                     ADE               0.9532  \n",
              "0                                     ADE               0.9641  \n",
              "0                                     ADE               0.9184  \n",
              "0                                     ADE               0.9998  \n",
              "0                                     NaN                  NaN  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-fdebf537-e412-43dd-b591-e7c8077caf79\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>assertion</th>\n",
              "      <th>entities_ner_chunks_ade_assertion</th>\n",
              "      <th>entities_ner_chunks_ade_assertion_class</th>\n",
              "      <th>assertion_confidence</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>allergic reaction</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9532</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>itchy</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9641</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>sore throat/burning/itchy</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9184</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>present</td>\n",
              "      <td>numbness in tongue and gums</td>\n",
              "      <td>ADE</td>\n",
              "      <td>0.9998</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fdebf537-e412-43dd-b591-e7c8077caf79')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-fdebf537-e412-43dd-b591-e7c8077caf79 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-fdebf537-e412-43dd-b591-e7c8077caf79');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-f8ec7b9f-8469-4b21-9d0b-501b657ad1c5\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f8ec7b9f-8469-4b21-9d0b-501b657ad1c5')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-f8ec7b9f-8469-4b21-9d0b-501b657ad1c5 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## nlu.autocomplete_pipeline(pipe)\n",
        "\n",
        "Auto-Complete a pipeline or single annotator into a runnable pipeline by harnessing NLU's DAG Autocompletion algorithm and returns it as NLU pipeline.\n",
        "The standard Spark pipeline is avaiable on the `.vanilla_transformer_pipe` attribute of the returned nlu pipe\n",
        "\n",
        "Every Annotator and Pipeline of Annotators defines a `DAG` of tasks, with various dependencies that must be satisfied in `topoligical order`.\n",
        "NLU enables the completion of an incomplete DAG by finding or creating a path between\n",
        "the very first input node which is almost always is `DocumentAssembler/MultiDocumentAssembler`\n",
        "and the very last node(s), which is given by the `topoligical sorting` the iterable annotators parameter.\n",
        "Paths are created by resolving input features of annotators to the corrrosponding providers with matching storage references.\n"
      ],
      "metadata": {
        "id": "txeCLCtWbSVp",
        "pycharm": {
          "name": "#%% md\n"
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from johnsnowlabs import medical\n",
        "text = \"\"\"I have an allergic reaction to vancomycin.\n",
        "    My skin has be itchy, sore throat/burning/itchy, and numbness in tongue and gums.\n",
        "    I would not recommend this drug to anyone, especially since I have never had such an adverse reaction to any other medication.\"\"\"\n",
        "\n",
        "re_model = medical.RelationExtractionModel().pretrained(\"re_ade_clinical\", \"en\", 'clinical/models')\n",
        "\n",
        "nlu_pipe = nlp.autocomplete_pipeline(re_model)\n",
        "df = nlu_pipe.predict(text)\n",
        "cols = [\n",
        "'relation_RelationExtractionModel_1fb1dfa024c7',\n",
        "'relation_RelationExtractionModel_1fb1dfa024c7_confidence',\n",
        "'relation_RelationExtractionModel_1fb1dfa024c7_entity1',\n",
        "'relation_RelationExtractionModel_1fb1dfa024c7_entity2',\n",
        "'relation_RelationExtractionModel_1fb1dfa024c7_entity2_class',\n",
        "]\n",
        "\n",
        "df[cols]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 601
        },
        "id": "Pk8ThwOaGHW2",
        "outputId": "e118b2f3-2271-45c8-9447-da54314ef26d",
        "pycharm": {
          "name": "#%%\n"
        }
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "re_ade_clinical download started this may take some time.\n",
            "[OK!]\n",
            "Warning::Spark Session already created, some configs may not take.\n",
            "embeddings_clinical download started this may take some time.\n",
            "Approximate size to download 1.6 GB\n",
            "[OK!]\n",
            "pos_anc download started this may take some time.\n",
            "Approximate size to download 3.9 MB\n",
            "[OK!]\n",
            "dependency_conllu download started this may take some time.\n",
            "Approximate size to download 16.7 MB\n",
            "[OK!]\n",
            "ner_jsl download started this may take some time.\n",
            "[OK!]\n",
            "sentence_detector_dl download started this may take some time.\n",
            "Approximate size to download 354.6 KB\n",
            "[OK!]\n",
            "Warning::Spark Session already created, some configs may not take.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  relation_RelationExtractionModel_1fb1dfa024c7  \\\n",
              "0                                             1   \n",
              "0                                             1   \n",
              "0                                             1   \n",
              "0                                             1   \n",
              "0                                             1   \n",
              "0                                             1   \n",
              "0                                             0   \n",
              "\n",
              "  relation_RelationExtractionModel_1fb1dfa024c7_confidence  \\\n",
              "0                                             0.5657         \n",
              "0                                             0.9604         \n",
              "0                                             0.9604         \n",
              "0                                             0.9604         \n",
              "0                                              0.667         \n",
              "0                                              0.667         \n",
              "0                                         0.75065005         \n",
              "\n",
              "  relation_RelationExtractionModel_1fb1dfa024c7_entity1  \\\n",
              "0                                  allergic reaction      \n",
              "0                                               skin      \n",
              "0                                               skin      \n",
              "0                                               skin      \n",
              "0                                              itchy      \n",
              "0                                              itchy      \n",
              "0                          sore throat/burning/itchy      \n",
              "\n",
              "  relation_RelationExtractionModel_1fb1dfa024c7_entity2  \\\n",
              "0                                         vancomycin      \n",
              "0                                              itchy      \n",
              "0                          sore throat/burning/itchy      \n",
              "0                        numbness in tongue and gums      \n",
              "0                          sore throat/burning/itchy      \n",
              "0                        numbness in tongue and gums      \n",
              "0                        numbness in tongue and gums      \n",
              "\n",
              "  relation_RelationExtractionModel_1fb1dfa024c7_entity2_class  \n",
              "0                                    Drug_Ingredient           \n",
              "0                                            Symptom           \n",
              "0                                            Symptom           \n",
              "0                                            Symptom           \n",
              "0                                            Symptom           \n",
              "0                                            Symptom           \n",
              "0                                            Symptom           "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-af5dc8f4-76c7-4e0d-8e24-3ca9cd907a95\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>relation_RelationExtractionModel_1fb1dfa024c7</th>\n",
              "      <th>relation_RelationExtractionModel_1fb1dfa024c7_confidence</th>\n",
              "      <th>relation_RelationExtractionModel_1fb1dfa024c7_entity1</th>\n",
              "      <th>relation_RelationExtractionModel_1fb1dfa024c7_entity2</th>\n",
              "      <th>relation_RelationExtractionModel_1fb1dfa024c7_entity2_class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0.5657</td>\n",
              "      <td>allergic reaction</td>\n",
              "      <td>vancomycin</td>\n",
              "      <td>Drug_Ingredient</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0.9604</td>\n",
              "      <td>skin</td>\n",
              "      <td>itchy</td>\n",
              "      <td>Symptom</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0.9604</td>\n",
              "      <td>skin</td>\n",
              "      <td>sore throat/burning/itchy</td>\n",
              "      <td>Symptom</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0.9604</td>\n",
              "      <td>skin</td>\n",
              "      <td>numbness in tongue and gums</td>\n",
              "      <td>Symptom</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0.667</td>\n",
              "      <td>itchy</td>\n",
              "      <td>sore throat/burning/itchy</td>\n",
              "      <td>Symptom</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0.667</td>\n",
              "      <td>itchy</td>\n",
              "      <td>numbness in tongue and gums</td>\n",
              "      <td>Symptom</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0.75065005</td>\n",
              "      <td>sore throat/burning/itchy</td>\n",
              "      <td>numbness in tongue and gums</td>\n",
              "      <td>Symptom</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-af5dc8f4-76c7-4e0d-8e24-3ca9cd907a95')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-af5dc8f4-76c7-4e0d-8e24-3ca9cd907a95 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-af5dc8f4-76c7-4e0d-8e24-3ca9cd907a95');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-740528a1-a17c-401e-a4bd-d5b81af15938\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-740528a1-a17c-401e-a4bd-d5b81af15938')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-740528a1-a17c-401e-a4bd-d5b81af15938 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "9P3-EmI0l6Dx",
        "pycharm": {
          "name": "#%%\n"
        }
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}